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Abstract 



Multi- Agent Systems (MAS) promise to offer solutions to problems where established, 
older paradigms fall short. In order to validate such claims that are repeatedly made 
in software agent publications, empirical in-depth studies of advantages and weaknesses of 
multi-agent solutions versus conventional ones in practical applications are needed. Climate 
control in large buildings is one application area where multi-agent systems, and market- 
oriented programming in particular, have been reported to be very successful, although 
central control solutions are still the standard practice. We have therefore constructed and 
implemented a variety of market designs for this problem, as well as different standard 
control engineering solutions. This article gives a detailed analysis and comparison, so as 
to learn about differences between standard versus agent approaches, and yielding new 
insights about benefits and limitations of computational markets. An important outcome 
is that "local information plus market communication produces global control" . 

1. Introduction 

When new paradigms arise on the scientific horizon, they must prove their value in com- 
parison and competition with existing, more established ones. The multi-agent systems 
(MAS) paradigm is no exception. In a recent book on software agents (Bradshaw, 1997), 
Norman observes that perhaps "the most relevant predecessors to today's intelligent agents 
are servomechanisms and other control devices" . And indeed, a number of applications for 
which multi- agent systems have recently claimed success, are close to the realm of what is 
traditionally called control engineering. One clear example is the climate control of large 
buildings with many office rooms. Here, Huberman & Clearwater (1994, 1995) have con- 
structed and tested a working MAS solution based on a market approach, that they reported 
to outperform existing conventional control. 

The key question studied in this article is: in what respect and to what extent are 
multi-agent solutions better than their (conventional) alternatives? We believe that the 
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above-mentioned application provides a nice opportunity to study this question in a de- 
tailed empirical way. It is practically very relevant, it lends itself to alternative solutions, 
and it is quite prototypical for a wide range of industrial applications in distributed re- 
source allocation (including energy management applications (Ygge & Akkermans, 1996; 
Akkermans & Ygge, 1997; Ygge & Akkermans, 1998), telecoms applications, the file alloca- 
tion problem of Kurose and Simha (1989), and the flow problems investigated by Wellman 
(1993)). 

This article gives a detailed analysis of a published MAS solution to building climate 
control, and it compares multi-agent markets with traditional control approaches. We also 
introduce an improved and novel multi-agent solution to this problem based on an equilib- 
rium market. From our comparative analysis we are able to draw general conclusions about 
the suitability of the various approaches. Briefly, we show how computational markets can 
be designed that perform as well as centralized controllers having global information about 
the total system. However, a major advantage of the market framework is that it achieves 
this in a fully decentralized fashion using only locally available data. We finally outline that 
it should be possible to come to a more general theory concerning the connections between 
markets and conventional control concepts. Here, we show that for the considered type 
of applications the quasi-equation "local data + market communication = global control" 
holds. 

The structure of the article is as follows. After an introduction (Section 2) to market- 
oriented programming and available theory, Section 3 gives the problem definition. Section 4 
introduces the application domain: it describes the office environment and gives the phys- 
ical model for cooling power and the various temperature and outside weather influences. 
We then discuss the results of a standard control engineering solution, based on local and 
independent integral controllers regulating the building climate (Section 5). Next, we re- 
view the market-based approach as put forward by Huberman & Clearwater (1994, 1995) 
(Section 6), and validate their claim that this market approach performs better than con- 
ventional independent controllers. We subsequently analyze this market protocol in detail 
and show that its success is related to the fact that the agents possess global information 
before the auction is started (Section 7). In Section 8 we then develop an improved stan- 
dard control engineering scheme that also exploits global data. Such a control scheme turns 
out to perform much better than the Huberman-Clearwater market. Finally, we propose a 
market design of our own based on general equilibrium theory. It performs as well as the 
controller having access to global data, but operates on local data only and thus represents 
a really decentralized solution (Section 9). Section 10 puts our results into perspective and 
summarizes the general conclusions in comparing different approaches. 

2. Market-oriented Programming 

The use of market mechanisms for resource allocation by computer systems has a rather 
long history in computer science, e.g. (Sutherland, 1968), and more recently markets have 
been used in a number of different application areas. 
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There is rather extensive theory available for the relations between optimization and 
markets. For example: 

• Jennergren (1973) has shown how a price schedule decomposition algorithm can be 
used for solving linear programming problems. 

• Kurose & Simha (1989) showed how an equilibrium constitutes an optimal solution 
to a file allocation problem. A similar result was shown by Bertsekas (1992) for the 
assignment problem. 

• Bikhchandani Sz Mamer (1997) showed that a Pareto efficient outcome 1 maximizes 
the sum of the utilities in an exchange market with agents holding quasi-linear utility 
functions 2 This is a significant extension to the above, as it also applies to non-linear 
problems. 3 

• The results of Bikhchandani and Mamer were extended to markets that include pro- 
duction and uncertainty by Ygge (1998, Chapter 3). The relations between these 
types of markets and traditional optimization problems were also made more explicit 
in the latter work. 

Markets approaches have been applied to a number of different computerized applica- 
tions, and some of them are briefly discussed here. Already in 1968 Sutherland proposed an 
auction mechanism for allocation of computational resources on a PDP-1 computer (Suther- 
land, 1968). Different amounts of money were assigned to different users in accordance with 
the importance of their projects. It was reported that the users then over time learn how 
to bid properly for computational resources. This basic approach has been further refined 
by, for example, Gagliano et al. (Gagliano et al., 1995). 

Kurose & Simha (1989) investigate a file allocation problem. In this work the agents 
report their marginal utility (for having a certain amount of storage) and its derivative to 
an auctioneer, which reallocates the resource using a resource-oriented Newton-Raphson 
algorithm (cf. Ygge & Akkermans, 1998) until an equilibrium has been reached in which 
the marginal utilities of all agents are the same. Although the paper is based on a number of 
microeconomic abstractions, it does not for example utilize prices and there is no trade-off 
between different commodities. So, it is debatable whether this approach can be referred 
to as really market-based. 4 

The assigning problem of allocating n objects to n users has been investigated by Bert- 
sekas (1992). Each user has a valuation of each object, and cannot be assigned more than 
one object. It is reported that each user can be seen as an economic agent, and it is shown 
how an auction (which essentially is an English auction) results in an equilibrium. The 

1. An allocation is Pareto efficient or Pareto optimal if there is no alternative allocation that makes any 
agent better off without making the outcome worse for some other agent (Varian, 1996, p. 15). 

2. An example of a quasi-linear utility function is given in Eq. (19). 

3. However, one should remember that this theory does not provide any computational advantages. If it is 
hard to find the allocation that maximizes the sum of the utilities, then it is also hard to find a Pareto 
optimal one (as they are the same in this case). 

4. One should also note that the formulation of the Newton-Raphson scheme of this approach is overly 
simplistic, and there are much better standard methods available (Press et al., 1994; Ygge & Akkermans, 
1998). 
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Figure 1: High-level view of an equilibrium market mechanism. 



assumed agent behavior is that an agent bids v — w for its most highly valued object (i.e. 
the object with the highest difference, v, between the valuation and the price), w is the 
difference between the valuation and the price of the second most preferred object. There 
is however no motivation given for the assumed agent behavior, and it is not clear why any 
agent would use this strategy unless it can be externally imposed. Though there is some 
economical interpretation of the prices, the approach is not very market-like due to the 
rather unrealistic assumptions of agent behavior. 

The equilibria in the applications by Kurose and Simha, and Bertsekas are proven to be 
optimal. It is not hard to see that the problem of Kurose and Simha can be reformulated 
as a proper market with quasi-linear utility functions, and that the competitive equilib- 
rium then is equivalent to the equilibrium described by Kurose and Simha, cf. (Ygge, 1998, 
Corollary 3.3.1). Similarly, the price vector obtained by Bertsekas clearly constitutes a 
competitive equilibrium. At the same time, it is shown by Ygge (Ygge, 1998, Theorem 3.2), 
that all separable optimization problems can be formulated in market terms and the (com- 
petitive) general equilibrium (if existing) is identical to the optimal solution to the original 
optimization problem. Consequently, this recent theory generalizes the earlier theory by for 
example Kurose and Simha, and Bertsekas. 

Wellman et al. have contributed significantly in developing market-based approaches to 
resource allocation into a programming paradigm, which they have given the name market- 
oriented programming, e.g. (Wellman, 1993; Mullen & Wellman, 1995; Wellman, 1995, 1996; 
Yamaki, Wellman, & Ishida, 1996; Hu & Wellman, 1996; Cheng & Wellman, 1998; Wellman 
& Hu, 1998; Walsh, Wellman, Wurman, & MacKie-Mason, 1998). Particularly, the micro- 
economic framework of general equilibrium theory has been successfully used as a resource 
allocation mechanism. In such a market which we call an equilibrium market agents send 
demand functions telling how much they like to consume or produce at different prices. 
The auctioneer then tries to establish an equilibrium price vector such that supply meets 
demand for all commodities, cf. Figure 1. 

The process of submitting parts of the demand function may be iterated if an equilibrium 
price is outside the region captured by the submitted demand functions. One such process 
is the basic price tatonnement process, cf. e.g. (Cheng & Wellman, 1998), in which demand 
functions for the respective commodities are sent to an auctioneer. Each of those demands is 
based on expected prices of the other commodities. That is, if those other prices change, a set 
of new demand functions may need to be submitted. Once the auctioneer has established 
an equilibrium price, the agents will exchange the resources as stated by their bid and 



304 



Decentralized Market Control 



the equilibrium price. (For example, if an agent states that it wants to buy 1/p units of 
a commodity — where p is the price in the commodity money — and the equilibrium price 
becomes 1, the agent will buy one unit of resource for one unit of money.) 

Equilibrium markets have many attractive theoretical properties. For example, if all 
agents act competitively 5 , the outcome is Pareto efficient. Furthermore, if all utility func- 
tions are quasi-linear, the outcome is globally optimal (Ygge, 1998, Theorem 3.2), and 
in the presence of uncertainty, the outcome maximizes the expected global utility (Ygge, 
1998, Theorem 3.5). We note that equilibrium markets be computationally implemented in 
a computationally very efficient manner (Ygge, 1998, Chapter 4). 

Wellman et al. have applied equilibrium markets to a number of applications, such 
as multi-commodity flow problems, design problems, and bandwidth allocation problems. 
They have also introduced a market-based approach to scheduling which has many similar- 
ities with the assignment problem of Bertsekas described above, but relies on more realistic 
assumptions on agent behavior. 

The present authors have introduced a market-oriented approach to power load man- 
agement (Ygge & Akkermans, 1996; Ygge, 1998; Ygge et al., 1999). 

We note that the aim of market-oriented programming in computer science is funda- 
mentally different from the aim of economic theory. This is visualized in Figure 2. In 
market-oriented programming, microeconomic theory is taken as given and serves as the 
theory for implementation of computational agents. Whether or not the microeconomic 
theory actually reflects human behavior is not the critical issue. The important question 
is instead how microeconomic theory can be utilized for the implementation of successful 
resource allocation mechanisms in computer systems. For example, even though no (or at 
least very few) people believe that humans use explicit utility functions when making their 
decisions, such functions do appear very useful to concisely represent human preferences for 
use in computational agents. 

It is obviously very interesting to investigate the use of computational markets for au- 
tomating trades between different self-interested parties, where information is private and 
is revealed only if there is an expected gain from doing so. However, the use of markets has 
also been proposed for standard resource allocation where the true utility/costs for all nodes 
that consume and produce resource is assumed to be available (though possibly uncertain 
and/or distributed in the system). The main arguments found in the literature for applying 
market to these types of problems are: 

• The numerous similarities between economic systems and distributed computer sys- 
tems suggest that models and methods previously developed within the field of math- 
ematical economics can serve as blueprints for engineering similar mechanisms in 
distributed computer systems (Kurose & Simha, 1989). 

• Auction algorithms are highly intuitive and easy to understand; they can be explained 
in terms of economic competition concepts, which are couched on everyday experi- 
ence (Bertsekas, 1992). 

5. An agent that acts competitively treats prices as exogenous, that is, the impact on the prices due to its 
own behavior is negligible (Varian, 1996, p. 516). This is a very reasonable assumption if the market is 
of at least moderate size and/or if there is uncertainty about the behavior of the other agents (Sandholm 
& Ygge, 1997). 
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Figure 2: A simplified view on the relation between economics and computer science with respect 
to microeconomic theory. Economists study humans and generate theory that is used 
for the explanation of human economic behavior. Computer scientists doing market- 
oriented programming use this theory as a basis for building working computational market 
systems. 



• Market approaches enable a natural decomposition, both from a software engineering 
perspective as well as from a computational perspective (Schreiber et al., 1999; Ygge, 
1998, Chapter 15). 

• Market approaches are very flexible in that they allow for ongoing addition and dele- 
tion of agents. No global changes are required — merely the demand/supply relation 
is altered (Ygge, 1998, Chapter 15). 

• Markets are informationally efficient in terms of information dimensionality (Jordan, 
1982), and the abstractions used are the most natural ones for the user (Ygge, 1998, 
Chapter 15). 

• The introduction of trading resources for some sort of money enables evaluation of 
local performance and valuation of resources, so that it becomes apparent which re- 
sources are the most valuable and which agents are using the most of these (Ygge, 



The above arguments are mainly conceptual and related to software/system design and 
engineering issues, and must prove their value from acceptance by software and system 
designers. But there are also different and in some respects stronger claims. For exam- 
ple, Huberman and Clearwater state (Huberman & Clearwater, 1995) for the application 
of building control that: "While in principle an omniscient central controller with access 
to all the environmental and thermal parameters of a building (i.e. a perfect model) could 



1998, Chapter 15). 
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optimally control it, in practice such knowledge is seldom available to the system. Instead, 
partial information about local changes in the variables (such as instantaneous office occu- 
pancy, external temperature, and computer use) is the only reliable source that can be used 
for controlling the building. " As an alternative to such an omniscient controller Huberman 
and Clearwater propose a market-based multi-agent approach to this problem. This raises 
a very interesting question: Are there applications in which the information structure is 
such that market-based approaches do better than traditional approaches? In this paper 
we will carefully examine this issue for a building climate control problem. We will inves- 
tigate what information different alternatives (traditional and new market-based) require, 
and what the performance of these different approaches are. We will use exactly the same 
problem formulation as Huberman and Clearwater in order to reproduce their results and 
evaluate other new approaches with their original formulation. This problem formulation 
is given in the next section. 

3. Problem Definition 

The task is to allocate a resource (cold air) in an office building, given the setpoint temper- 
atures of the respective offices. As a measure of the success of the allocation, the standard 
deviation of the deviation from the setpoint is used (Huberman & Clearwater, 1995), i.e. 



where (•) denotes the average value of a variable, T% is the actual temperature of an office, 
and T^ etp is the setpoint temperature. The index i denotes a time interval under observation 
and the index o denotes the office under observation. This naming convention is used 
throughout the article. (In addition the index k will also be used to denote time periods.) 

It may be debated whether or not this is the best measure, but we stick to it in this 
article in order to evaluate the approach taken by Huberman and Clearwater using their 
own measure. 

4. The Office Environment 

The case study we consider for the comparison of decentralized markets versus central 
control solutions is the building climate control of a large office environment. 

In this section, we present a mathematical-physical model of the office environment. 
We first give a conceptual summary so that it is possible to understand the basic ideas of 
the model without studying the equations. The offices are attached to a pipe in which the 
resource (cold air) is transported as in Figure 3. The characteristics of this system are similar 
to the characteristics of a district heating system, but with offices instead of households. 
We assume that there are 100 offices in total, and that they are equally distributed towards 
East, South, West, and North. 

The thermodynamics of the office environment is actually quite simple. Every office is 
seen as a storage place for heat, but heat may dissipate to its environment. In the model, 
the thermodynamic behavior of an office is equivalent to a basic electrical RC-circuit. Here, 




(1) 
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Figure 3: Offices and air transportation. 



voltage is analogous to temperature, and electrical current is analogous to heat flow. C and 
R then respectively denote heat capacitance and thermal resistance. 

A good general reference on thermodynamic models as the one we use here is the book 
by Incropera & Witt (1990). The heat equations are continuous in time, but are discretized 
according to standard procedures from control engineering, cf. (Ogata, 1990). The ontol- 
ogy and reusability aspects involved in thermodynamics model construction are discussed 
extensively by Borst, Akkermans, & Top (1997). 

4.1 Thermodynamic Properties 

The resource treated is cooling power. Each office can make use only of a fraction, rj, of the 
available resource at that office, P^ ail , so that 

peons ^ pavail fn\ 
r io -i '/ ' r jo > \ L ) 

where P^ ns is the consumed power. The available resource at one office is equal to the 
available resource at the previous office minus the consumed resource at the previous office. 
Throughout this article we assume an r\ of 0.5. 

We treat everything in discrete time. The time interval we use in the calculations is 
one minute. For all offices the temperature, Tj G , is obtained by integrating a differential 
equation in discretized form: 

T w = T , o + £ (Pko at ~ PkD/Co, (3) 
fe=l 

where P^ at is the heating power and C a is the thermal capacitance. The heating power is 
described by 

p^t = {T virt_ T . o)/Ro ^ (4) 
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where R Q is the thermal resistance and T^ rt is a virtual outdoor temperature, described in 
more detail below. 

From Eqs. (3) and (4) we see that there is a feedback loop between the office temperature 
and the heating power. Solving for the temperature we obtain 

(rpvirt \ 
1 io _ peons \ 
Ti-i,o+ R ° l ° ],i>0. (5) 

This is the equation for the dynamics of the system that can be directly computed. At the 
right-hand side we have known quantities, where C Q and R D are externally given building 
parameters, Pf° ns is the output of the utilized controller (as will be described in detail later 
in the article for various different controllers), and T^ rt is obtained from the weather model 
below. 

4.2 Weather Model 

All external weather influences on the office environment are modeled by a virtual tem- 
perature, representing the outdoor temperature, sun radiation, etc. We assume that there 
is sunshine every day and that the outdoor temperature, T outd , varies from 22 to 35° C 
according to 

rpoutd = 22 + 13 • e^'-^' 5-4 -' mod 24 ~ 12 ) 2 / 20 (g) 

where s is the length of each time interval expressed in hours, so here s = 1/60. 
The virtual temperature, T^ rt , is described by 

rpVirt rpOUtd _|_ rpSUn _|_ rpfluct /y\ 

1 io d i ' 1 o' 1 ioi \V 

where T^ luct is a random disturbance, thought to represent small fluctuations caused by for 
example the wind. T^ luct is Gaussian distributed with zero mean and a standard deviation 
equal to unity. T sun is the sun radiation component. For the offices located at the East 
side T sun is described by 

rpSun,East _ g _ e _((j. s +4) mod 24-12) 2 /5 ^ 

and correspondingly for the South and the West offices 

rpsuu, South _ ^ _ g _ (*' s mod 24-12) 2 /5 ^g-j 

and 

rpSun,W est _ g _ g — ((i-s-4) mod 24— 12) 2 /5 (10) 

The various temperatures are plotted in Figure 4. 

4.3 Office Temperatures without Control 

In Figure 5, the temperature for a South-oriented office is plotted with different thermal 
resistances, R Q , and thermal capacitances, C Q . For simplicity we assume all R Q to be equal 
and all C Q to be equal. From this figure we observe two things: first, the higher R C 
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Figure 4: The plot at the left shows the outdoor temperature, T° utd . The middle plot shows the sun 
radiation components, T sun , (with peaks at time 8, 12, and 16 hours for offices located 
at the East, South, and West sides, respectively) . Finally, the outdoor temperature plus 
the sun radiation components are plotted at the right. 
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Figure 5: The indoor temperature for an uncontrolled office is plotted for different values of the 
thermal resistance and the heat capacitance. Small values of the thermal resistance and 
capacitance give higher peaks, while higher values give smoother curves. 



the bigger the lag in temperature changes, and second, the higher R C the smaller the 
fluctuations in temperature. For the simulation experiments in this article we took R Q = 10 
and C = 10. An R Q or a C Q equal to zero implies that Tj G = T^ rt , as can be seen by 
letting R or C approach zero in Eq. (5). Clearly, without control the office temperatures 
strongly fluctuate and reach unacceptably high values in all cases. 
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5. Control-A: Conventional Independent Controllers 
5.1 Integral Control for the Offices 

The application of regulating office temperature has a long tradition in control theory. The 
most widely known controllers are different variants of the PID controller. The letters PID 
denote that the control signal is proportional (P) to the error (that is, to the difference 
between the setpoint and the actual value); proportional to the integral (I) of the error; or 
proportional to the derivative (D) of the error. Here, we use a variant of an integrating 
controller 6 of the form 



where F is the output signal from the controller, and (3 is a so-called gain parameter (that 
can be set externally in the design of the controller). For the simulations it is assumed 
that Fi is limited to a value between zero and three. This is in order to model that the 
valves can be closed but that cooling resources are not delivered from one room to another 
(the lower bound), and that there is a maximum to the amount of resources that can be 
obtained by opening the valve fully (the upper bound). The control signal is sent to the 
actuator and the actual Pf° ns is obtained from 



Plots of the office temperatures with different gains are shown in Figure 6. The gain of 
the controller is not critical for this application. Too high a gain will result in the controller 
overreacting when the temperature exceeds the setpoint, after which it will be under the 
setpoint for quite some time. This leads to a larger error than if smaller adjustments are 
made. Also, the amplitude of the control signal then gets unnecessarily high, but the system 
does not get dangerously unstable. We note that the maximum deviation here is ±0.06°C. 
Thus, controllers using any of the three gains perform very well. In the further calculations 
of this article a gain equal to 10 has been adopted. 

5.2 The Implications of Limited Resources 

So far, we have assumed that the total amount of available resources is unlimited. Now, 
we suppose that there is a maximum value for the cooling power that is inserted into the 
system. In such a situation, offices that are situated close to the air input will obtain a 
sufficient amount of cool air, but those near the end will suffer if totally uncoordinated 
controllers are used. Thus, the smaller the total amount of available resources, the larger 
the standard deviation will be. This is visualized in Figure 7. As a reasonable figure we 
have chosen an upper limit for the total resource amount of 140. 

We conclude, as shown by the example, that independent integrating controllers perform 
very well when the amount of cooling resources is unlimited. On the other hand, when there 

6. The main reason for choosing an integrating controller is that we want to have a setting that is as close 
as possible to the setting described by Huberman and Clearwater (1995). Other controllers for this 
application may well be considered, but as long as the performance is as good as shown in Figure 6, this 
is not at all crucial for the central argument of this article. 



Fi- l!0 + P(T i0 -T° et P), 



(11) 



10 




(12) 
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Figure 6: The indoor temperature for an office, utilizing an integral controller, is plotted for different 
controller gains. The setpoint temperature is 20°C. 




Figure 7: The standard deviation (in °C) , as defined in Eq. (1) for each interval, i, is plotted 
for different amounts of available cold air resources (130, HO, 150, and 160) with 100 
offices. The lower the amount of available resources, the higher the standard deviation. 
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is a shortage of available resources, the standard deviation increases dramatically, yielding 
poor performance. 

6. Market-A: The Approach by Huberman and Clearwater 

A multi-agent systems solution to the problem of building control has been presented by 
Huberman and Clearwater (1994, 1995). The approach taken is to model the resource 
allocation problem of the building environment as a computational market where agents 
buy and sell cooling power resources. The non-separability in terms of agents is ignored. 7 
The basic idea is that every office is represented by an agent that is responsible for 
making good use of its resources, and to this end trades resources with other agents. The 
agents send bids to an auctioneer, which calculates a clearing price, and makes sure that 
no agent has to buy for a price higher than its bid nor has to sell for a price lower than its 



In this section we give the market protocol proposed by Huberman and Clearwater, and 
we reproduce their results. In order to make the paper self-contained, the interested reader 
can find the original equations underlying the Huberman-Clearwater market protocol in 
Appendix A. 

6.1 Market protocol 

A bid is constructed as the following tuple: 



where sell is a boolean variable indicating whether the current bid is a sell bid (true) or 
buy bid (false), v is the volume traded for, and B is the price that the agent demands (sell 
bid) or is prepared to pay (buy bid). 

Each of the variables in Eq. (13) is a function of other variables according to 



where Tj = [Tn, Tj2, • • • , Tiioo]> T setp = [T^ e p , T% e p , . . . , T^q p ], a is a strength parameter of 



the auction, and rrii is called a money parameter. The trade volume is directly proportional 
to q and, hence, doubling a doubles the traded volume in an auction. Thus, a is a free 
parameter that can be externally set to tune the trade volumes to proper levels. The 
money parameter does not correspond to any real currency; it is merely a control variable, 
varying between 100 and 200, and it does not have any direct connection to, for example, 
the value of the cooling resource. The money cannot be saved between auction rounds 

7. A resource allocation problem is separable if the total utility of the system of the system can be expressed 
as the sum of the utilities of each node (producer/consumer), and the only constraints on how much 
resource that can be assigned to each node is only restricted by that their sum must not exceed the total 
available resource (Ibaraki & Katoh, 1988). Hence, the problem treated in this paper is not separable as 
there are other constraints on how the resource can be distributed. For example, if we assign sufficiently 
much to the penultimate agent, the ultimate will have a significant amount of resource available. 



bid. 



bid = [sell io ,v io ,Bi ], 



(13) 



selli — selli (Ti ,T ^, {T^, (T Q p )), 

v io = v io (T i ,T set P,a),and 

Bio = Bi {Ti 0l T (Ti) , (T p ),T7ij ), 



(14) 
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in the Huberman-Clearwater protocol, another indication that it simply functions as an 
adjustment parameter in this protocol. 

An auction consists of the following three phases: 

1. All agents send their bids, [selli (T io ,T* etp , (Tj), (T* etp ), v io {Ti, T setp , a), 
B w {T l0 ,T s etp , (Tj), {T s etp ),m l0 )], to the auctioneer. 

2. The auctioneer then computes a market clearing price by searching for the following 
minimum: 



mm 

Pi 



X] Vi °~ X! Vi ° 

o\sell=true,Bi <pi o\sell= false, Bi a >pi 



(15) 



This yields a price such that supply matches demand as closely as possible. 



3. Finally, the resources are allocated as dictated by the above bids and the obtained 
market price, according to the rule: all sellers that are offering Vi Q for a price, Bi Q , 
that is lower than or equal to the market price will sell V{ ; and correspondingly for 
buyers. 8 

As an example of the computation of the equilibrium price, assume that the auctioneer 
has received the following bids [true, 2,4], [true, 1,3], [true, 2, 2], [false, 1,3], [false, 2, 2], 
[false, 2, 1]. Then the market clearing price is 3, leading to acceptance of bid 2, 3, 4, and 5. 
(Bid 1 is too high a sell price and bid 6 is too low a buy price.) 

6.2 Simulations 

Figure 8 shows two plots of a simulation for the period between 3 p.m. and 7 p.m. 9 The 
initial temperatures for all offices were set to 20° C. The upper plot is the standard de- 
viation when independent integrating controllers are used, and the lower one shows the 
agent-based control scheme as defined above. We found that for the adjustment parameter 
a (see Eq. (23)) a value of 64 led to the smallest overall standard deviation. The key ob- 
servation from the figure is that the agent approach offers at least one order of magnitude 
of improvement. 

Compared to independent conventional controllers this is indeed a major advance, and 
this is the central claim made by Huberman and Clearwater. The market approach given 
above leads to a reduced control error and thus to increased comfort compared to stan- 
dard controllers. Our simulations thus reproduce and validate the results of Huberman & 
Clearwater (1994, 1995). 

8. Since the bids are given using discrete volumes, supply will seldomly match demand exactly at the 
clearing price. Normally, there will be a very small excess demand or supply. If there is an excess supply, 
all buyers that are willing to pay at most the clearing price will buy, but not all sellers that are willing 
to accept at least the clearing price can sell. In this situation, a seller is selected randomly from the 
valid candidates does deliver only a fraction of its bid. A corresponding procedure is used when there is 
a small excess demand. 

9. All solutions discussed in this article have been implemented in and simulated by the authors using 
C++ on a PC running Windows95. Furthermore, all simulations have been independently recreated and 
verified by Bengt Johannesson from various relevant papers (Ygge & Akkermans, 1997; Huberman & 
Clearwater, 1995; Clearwater & Huberman, 1994)), as part of a master's thesis project, using Python 
on a PC running Linux. 
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Figure 8: Standard deviation for independent controllers (top) and agent-based control (bottom). 

However, this cannot be the last word in a successful system study. A first question to 
be raised is whether we can find out in more detail what the actual reason is for the success 
of the above market-based approach. A second relevant question is whether there are 
alternative, market-based and/or central control-based, solutions to the building control 
problem that are even better. The answer to both questions is yes, as we set out to 
demonstrate now. 

7. Market-A': A Suite of Variations 

In this section we present a suite of variations on the scheme presented in Section 6. The 
main aim is to understand which of many factors involved are actually responsible for the 
good performance of the Huberman-Clearwater market approach. 

7.1 Deleting the Money Dependency 

A first simplification is to remove the adjustment parameter called money. This is done by 
setting rrii to a very high value, C, regardless of the resource allocation. 10 The simplified 
market protocol then is: 



1. All agents send their bids, [sell lo (T io , T^, (T 4 ), (T^), v lo (T h T set P,a), 
B to {T io , T setp , (T t ), (T° et P),m i0 = C)}, to the auctioneer. 



2. The auctioneer computes a market price from 



mm 

Pi 





(16) 



o\sell=true,Bi <pi 



o\sell=false,Bi >pi 



3. Allocate the resource as dictated by the bids and the market clearing price. 
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Figure 9: Standard deviation for the original Huberman- Clearwater scheme and a scheme without 



Plots of the standard deviation with the original scheme as well as with a scheme where 
all money dependencies have been removed are shown in Figure 9. For the scheme without 
money, setting the a parameter to 66 turned out to be optimal. The scheme without 
money dependencies performs as well as the original scheme. The reason for this is that 
.Bjo's dependency on mj e is completely negligible. 11 Hence, the money parameter does not 
play any significant role in the success of the Huberman-Clearwater market scheme. 

7.2 Deleting the Temperature Dependency 

Another factor of interest is the impact of the temperature on the bid prices. We remove 
this dependency by setting the bid price to 10 for all selling agents and to 100 for all buying 
agents. That is, we now use the following market scheme: 

1. All agents send their bids, 



to the auctioneer. 

2. Now, all prices larger than 10 and smaller than 100 are equally good candidates. If 
there is a mismatch between supply and demand, say, supply exceeds demand, the 
agents that will sell are picked randomly among the valid candidates, and the resources 
are allocated accordingly. 



10. More precisely, this is done by setting (7(0, m) in Eq. (28) (Appendix A) to a constant value, 2000. This 
corresponds to letting m io approach infinity. 

11. In actual fact, U(0, m) in Eq. (28) of Appendix A will vary only between 1999.86 and 2000 for all possible 



any money dependencies. 



[sell io (T io , T^p, (Ti), (Tf ^(T,, T set P, a), B, 




sell = true 
sell = false 



mio. 
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Figure 10: Standard deviation for the scheme with the money dependencies removed, compared to 
a scheme with both the money and temperature dependencies removed. 



In Figure 10 the standard deviation is plotted for the scheme without money, mentioned 
above, and for a scheme where the dependency on the temperature has been removed as 
well. Here, setting the fit parameter a to 65 turned out to be optimal. We see that also 
here the performance is as good as that of the original scheme. Note that the temperature 
is still used to determine both V{ and the boolean variable sell. But the conclusion is that 
the temperature dependency does not affect the quality of the market-based solution to the 
building control problem. 



7.3 Deleting the Auction 

Next, we let the agents assign their bids to themselves without any auction whatsoever. This 
means that the sum total of the controller outputs, Fi Q , might sometimes exceed the total 
resource amount and sometimes be below that. The physical model is of course still obeyed, 
so that the total resource amount actually used, the total consumed cooling power Pi° ns , 
will never exceed the totally available resource amount, as described by Eq. (2). Hence, the 
resource updates are described by: 



F' 

to 



Ft. 



F' 

3, 



+v i0 (Ti, T set P, a), sell = false 
-v io (Ti, T set P, a), sell = true 

FL>3 



(17) 



The result of this simulation is shown in Figure 11. Here, an a of 17 turned out to be 
optimal. The surprising conclusion is that the multi-agent scheme without performing any 
auction is roughly ten times better than the auction-based Market-A scheme. 12 

12. From a practical point of view this is a pathological solution, revealing a loophole in the problem 
definition of Clearwater and Huberman, cf. Section 3. The revealed loophole of the problem definition is 
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Figure 11: Standard deviation with the auction mechanisms completely removed, compared to the 
original Huberman- Clearwater Market- A scheme. 



7.4 Discussion 

At first glance, it might seem counterintuitive that performance actually improves signifi- 
cantly when the core mechanisms of a market are removed. First we showed that introducing 
the market improves performance considerably compared to conventional independent con- 
trol, and then we showed that market mechanisms are superfluous. What, then, is the big 
difference between the uncoordinated integrating controller, the Control-A scheme, and 
the multi-agent scheme without any auction that we ended up with? The simple answer, in 
our view, is the access to global data that each agent has, in terms of the average temper- 
ature and the average setpoint temperature (as follows from Eq. (14)), in the Market-A 
and Market-A' schemes. Knowing average quantities means that an agent has some infor- 
mation about the other agents c.q. offices. This information is not available in the case of 
independent conventional controllers. The exploitation of this non-local information makes 
the big difference. 

that it does not take absolute temperature into account, but only the differences between temperature 
(i.e. minimizing standard deviation while potentially sacrificing average temperature). In the proper 
approaches introduced in the remainder of this article we do not allow ourselves to take advantage 
of this loophole, but minimize differences while using all available resource. That is, all approaches 
that are introduced from now on solve a problem which is strictly harder than the problem defined 
in Section3. What happens in the Market-A scheme without an auction can be seen from observing the 
definition of t io in Eq. (22). It is based on a quotient between the setpoint temperature and the actual 
temperature. Then the request volume, Eq. (24), is proportional to this quotient. This always results in 
a significant excess supply. (Hint, think of a small example with two offices, both with setpoint 20, and 
with temperatures 19 and 21. Now, || is farther away from 1 than |y, and hence the selling volume will 
exceed the buying volume.) This implies that at each trade occasion, many offices are prevented from 
selling (because of poor definitions of the volumes). The offices that are prevented to sell will deviate 
significantly, and hence this results in a high value of the measure. When the auction is deleted, all 
agents that are below the average distance to the setpoints are now able to get rid of the resource and 
hence the differences in temperature will decrease. 
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Thus, the upshot of our factor analysis of the success of the Huberman-Clearwater 
market scheme of Section 6 is that the agents have to share crucial office and control 
information first, before the auction takes place. This is why one can remove the auction 
without any deterioration of the results: a major function of an auction is to share necessary 
information through the bidding procedure, but in the Huberman-Clearwater protocol the 
needed information has already been shared. A more appropriate description of this market 
scheme would therefore be to say that it consists of not three (as described in Section 6), 
but four phases, in which the first phase is that all agents send their current temperatures 
Ti as well as their setpoint temperatures T£ etp to all other agents. 

In sum, we have now answered the first question raised at the end of the previous section: 
What is the reason for the success of the Huberman-Clearwater market scheme, compared 
to conventional independent controllers? The single most important factor is contained in 
the first step above: before the market procedure is carried out, the agents communicate key 
information about their current situation to all other agents. If that is done neither money 
nor auction mechanisms are needed. For conventional independent controllers, with which 
Huberman and Clearwater compare their own approach, this globally shared information is 
not available, however. This explains why these two solutions perform so very differently. 

Now, we turn to the second question raised in the previous section: are there alternative 
and perhaps better solutions? The above analysis suggests that we look into two different 
directions. If one wants to persuade control engineers of the value of agent-based approaches, 
it will be more convincing to compare the Huberman-Clearwater scheme not to strictly local 
controllers, but to controllers that have access to the same global information as the agents 
have. Secondly, this dependency on global information by the agents is a highly undesirable 
feature that clearly runs counter to a decentralized computational market philosophy. So, 
one should investigate whether it is possible to design a market protocol that exploits only 
strictly local information in the agents' bid construction. These issues we investigate in 
the following two sections, first treating central control and subsequently devising a strictly 
local market protocol. It will be seen that both yield significantly improved performance. 

8. Control-B: A Standard Controller Using Global Data 

Having concluded that the access to global data is crucial for performance, it is of course 
of interest to analyze what the performance would be of an integrating controller, like the 
one introduced in Eq. (11), but now incorporating global data. 

We would like the controller to take into account not only its own deviation from its set- 
point, but to consider also the deviations of the other offices from their setpoints. Therefore, 
the controller in Eq. (11) is extended to 

F io = Fj_i ;0 + 0[(T io - T seip ) - ((T t ) - (T set *>))}, (18) 

where (5 is set to 10 as previously. P^ ns is, as before, obtained from Eq. (12). 

The plot from the simulation with this controller, compared to the Market-A sim- 
ulation, and to the Market-A' simulation where the auction was removed, is shown in 
Figure 12. 

We see that the standard deviation is approximately the same for the Control-B and 
the Market-A' schemes. Thus, the Control-B scheme also performs roughly ten times 
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Figure 12: Standard deviation with an integrating controller that utilizes global data compared to 
the Huberman- Clearwater Market- A scheme, and to the Market- A' scheme with the 
auction removed. 



better than Market-A. An important difference, though, is that Control-B employs a 
well-known integrating controller for which there are well-understood methods and theories 
for e.g. stability analysis. 13 In contrast, the Market-A scheme is not easily analyzable 
from a formal theory perspective, since it does not rely on such well established concepts. 
While Control-B is relatively clear, see Eq. (18), it is much harder to conceptually grasp 
the principles behind Market-A, as it consists of a large number of complicated and not 
easily justified equations (see Eqs. (21)-(33) in Appendix A). The main conclusion from 
the present section is that, if we enable conventional controllers to act upon the same global 
information as the Huberman-Clearwater agents do, the central control approach performs 
best and is easiest to understand. 

9. Market-B: A Market with Local Data Only 

Thus, the computational market Market-A is outperformed by the global control scheme 
Control-B. Therefore, an interesting follow-up question is whether a simple and well 
performing computational market approach can be devised that does not depend on hav- 
ing available global information, in contrast to both these schemes. In this section we 
show that this is indeed the case, such that a decentralized market performs comparably 
to fully centralized conventional control. The approach derived in this section relies on 
more general theory for the relations between optimization problems and markets- (Ygge, 
1998, Chapter 3) and (Bikhchandani & Mamer, 1997) - applicable to this and many other 
applications. 



13. This holds under the assumption that the characteristic time scale of the variations of the average 
temperature is much larger than that of the fluctuation of the temperature. This is a very reasonable 
assumption in the present case. 
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We use a market-oriented programming approach based on an equilibrium market. The 
basic protocol for this is (cf. Figure 1): 

1. Each agent submits a net demand function Zi {p) to the auctioneer, describing what 
change in allocation is desired by agent o at price p. 

2. The auctioneer computes an equilibrium, by calculating a price pi such that 
2~2 z io(Pi) = (or, strictly speaking, \J2 z io(pi)\ < e > where e is a small (positive) 
numerical tolerance). 

3. Each agent receives its demanded resource Zi {pi) as calculated at the obtained market 
equilibrium price. 

The computation of Zi Q {p) and the process of computing the equilibrium are described 
further below. Note that this is a truly distributed approach, as it does not rely on any 
global data, or communication of temperatures between the agents before the market com- 
munication starts. 

9.1 The Relation Between Markets and Conventional Controllers utilizing 
Global Data 

The performance measure for the system is given by Eq. (1). The best system is therefore 
one that minimizes this equation. Hence, the most straightforward move one can think of 
to come to a market model, is to take this measure as representing the utility function for 
the overall system. So, the utility functions for the individual agents are ideally related to 
[(Tio — T£ etp ) — ((Tj) — (T setp ))] 2 . 14 However, this is still a formulation containing global 
information. 

Thus, we want to obtain a purely local reformulation, by getting rid of the terms with 
(T) containing the global information. We might replace them, though, by terms relating 
to the changes in the local resource. In doing so, we take inspiration from the standard 
controller equations Eqs. (11) and (18), indicating that we get good results (for uncon- 
strained resources) with the update equation Fi = i^_i i0 + (f>io, when <pi has the form 
0«o = P(Tio — T£ etp ). The intended interpretation of 4>i is to represent the output that the 
local controller would have delivered if it acted independently with unconstrained resources. 
In the market setting each agent updates its control signal Fj c through Fi Q = + AFj , 

where AFi Q is determined by the outcome of the market. Since the resource is only re- 
distributed among the agents, we have that J2o=i ^Fio = 0- Accordingly, as a step in the 
design of a Market-B scheme, based on local data only, we employ the following definition. 

Definition 9.1 Let the utility function for the individual office agents be defined by 

u(AF io , m) = -a 2 a (AF io - (j) io ) 2 + m, (19) 

where a Q is a strength parameter for each office representing its physical properties such as 
R and C Q . (The proper choice of a Q is discussed in Theorem 9.3 later on.) 
Furthermore, let all agents be price-takers. 

14. Since utilities are expressions of preference orderings, they are invariant under monotonic transforma- 
tions. 
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Due to the simple quadratic form of the utility function in Eq. (19), the net demand, 
Zi (p), is easily computed for price-taking agents by analytical means. Note that the money 
m here are fundamentally different from the money of Market-A, c.f. Section 6. Here the 
money represents a commodity and the agent needs to decide how much money to trade 
for the commodity cold air. As described above, the money in Section 6 is just a control 
variable without an intuitive interpretation. 



Theorem 9.1 With the utility functions and behavior given by Definition 9.1, there exists 
a unique equilibrium price pi \ ^2i zioipi) — 0; and the allocation obtained by the equilibrium 
market is Pareto optimal. 



Proof. See (Mas-Colell, Whinston, & Green, 1995; Varian, 1992; Takayama, 1985). □ 

In this setting, there is a wide variety of different algorithms that are guaranteed to 
converge to the clearing price Pi\J2o z io(Pi) = 0) m t ne second step of our equilibrium 
market protocol given above. Examples of such algorithms are price tatonnement, Wal- 
RAS (Cheng & Wellman, 1998), and various Newton- Raphson type methods. A detailed 
discussion of communication and computation efficiency aspects of such algorithms is given 
by Ygge (1998, Chapter 4). 



Theorem 9.2 Any Pareto optimal outcome in a market in which the agents hold the utility 
function of Eq. (19) is equivalent to an integrating controller that exploits global infor- 
mation as described by the following resource update equation : 

AF io = ct> io - — (</>i), (20) 
aj ■ (l/a 2 ) 

given that no agent is at its boundary. Here, (1/a 2 ) and ((pi) are the average \jo? , and 4>io 
respectively. 



Proof. At a Pareto-optimal allocation where no agent is at its bounds, all 

du(AFi )/dAFi are equal. (Assume du(AFi p ) / 'd AFi p < du(AFi q ) / dAFi q , then at any 
price du(AFip) / d AFi p < p < du(AFi q ) / dAFi q , there exists a (sufficiently small) amount 
of AFi, say e, such that u p (AFi p — e,mi p + pe) > u p (AFi p , mi P ) and u q (AFi q + e,mi g — 
pe) > u q (AF iq ,m,i q ). Correspondingly for du(AF ip )/dAF ip > du(AF iq )/dAF iq . Hence, 
du(AF ip )/dAF ip = du(AF iq )/dAF iq ). 

2 

Thus, it will hold for every office that l\Fi — <fti = -4-(AFiN — 4>in)- Summing the 
equations yields J2o=i ^ F io~J2o=i 4>io = a 2 N (AF iN -(j) iN ) J2o=i ■ Adding AF iN -<p iN to 
both sides and dividing both sides by together with the resource constraint (J2o=i ^-Fio = 

_yN 

0) yields n 1 = ot 2 N {AFi^ — fax) — °~ N °° which is equivalent to AFjjy = 4>iN — 
o? -(l/a 2 ) ^ or reasons °f symmetry, this equation holds for all offices. □ 

Corollary 9.1 For the special case where alla Q are equal, and(j)i = l3{Ti —T^ etp ) , Eq. (20) 
becomes exactly the Control-B scheme captured by Eq. (18). 
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Consequently, it also follows that simulation results for an equilibrium Market- B 
scheme, based on the utility function of Eq. (19) with the provisions given in the above corol- 
lary, are identical to those of the centralized Control-B scheme, cf. Figure 12. This fully 
decentralized market scheme thus performs better than the original Huberman-Clearwater 
market protocol. 

In sum, we see that (under suitable conditions which are fulfilled here) local data plus 
market communication is equivalent to conventional central control utilizing global data. 
Even though our proof was based on the assumption that the agents are never at their 
boundary values, it will be a close approximation in many practical applications. It should 
also be noted that managing the boundaries is not required for a successful implementation. 
As seen above, omitting the management of boundaries in the current application leads to 
Control-B, which was shown to have a very high performance. 

9.2 Finding an Optimal Utility Function 

In this section we show how an optimal utility function is constructed in the constrained 
case, from an optimal controller for the unconstrained case. 15 Earlier we noted that it is 
debatable whether or not the measure in Eq. (1) is a good one. One argument against it is 
that it allows for pathological solutions, as it does not take the average indoor temperature 
into consideration. If we for example increase the indoor temperature in every office by 
10°C, all of them could be unbearably hot, while the measure might be minimized. A 
reasonable thing to do here, to prevent these kind of solutions, is to treat the average 
temperature as given (the result of using all available resources). If we do so, the following 
theorem shows how an optimal utility function is constructed in the constrained case, from 
an optimal controller for the unconstrained case. 

Theorem 9.3 // Ti is a linear function of AFi Q ,^ and if (f>i minimizes Eq. (1) in the 
unconstrained case, then in a market where the utility functions are described by Eq. (19), 
with a Q = fr£p. , any associated Pareto optimal allocation minimizes Eq. (1) in the con- 
strained case, if the resource can be independently allocated among the agents (i.e., when 
the only constraint on AFi Q is the upper and lower bound, and J2o=i ^Fi = 0), and the 
average temperature is considered as given. 

Proof. Minimizing Eq. (1) boils down to minimizing /(Tji,Tj2, . . . , Tj n ) = Y^o=\[^io ~ 
T^ etp ) — ((Tj) — (T setp ))] 2 , since this is just a monotonic transformation. Due to the fact that 
4>io minimizes Eq. (1), it holds that d ^ F . = for AFi = <pi Q . Since the average temperature 

is considered as given, we have that = 2[{T io {4> lo ) - T^) -«!})- (T setp ))] • gg^ 

for AF io = <t> i0 . This gives that T to (&o) = T^> + ((T t ) - (T set P)). Thus, / can be 
rewritten as J2o=i[ T io{^ F io) 

— Tj (^>j )] . Since Tj G is a linear function of Ai^ , we have that 
T io {AF io ) = T^io) + • {AF W - <f> io ), and hence / becomes £li (^fc) 2 (AF to - 

4>io) 2 . As the reallocation of m does not effect the total (summed) utility, minimizing 

15. The resource is constrained when the action of any agent is limited by Pi , see Eq. (12) and Figure 7. It 
is unconstrained when such a limitation is not present, cf. Figure 6. 

16. From the thermal model discussed in Section 4 (especially Eqs. (5) and (12)) we note that this is indeed 
the case for a reasonably wide range. 
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/ is equivalent to minimizing — J2o=i u o, with a = g A jff . Furthermore, we have that 
any Pareto optimal allocation in a market where the utility functions are described by 
Eq. (19) maximizes J2o=i u o- (Suppose that there is an allocation AF^ Q that does not 
maximize J2o=i u oi but that AF^ does. Then reallocating AFi to AF^ and letting m b io = 
mf + u (AF<* , mf ) - u a {AF^ , m\ ) + ^ MAF£,,<)-£ o=1 u (AF? o , m « o ) . g & pareto 

improvement. Hence, a non-maximized sum implies a non-Pareto optimal allocation, and 
a Pareto optimal allocation implies a maximized sum.) Then (Ygge, 1998, Theorem 3.1, 
p. 44) implies that any Pareto-optimal allocation in a market where the utility functions 
are described by Eq. (19) with a Q = , minimizes /. □ 

Thus, we have constructed a fully distributed market design that yields an optimal 
outcome. Particularly note that Theorem 9.3 was not based on the assumption that an 
integral controller was used, rather it said that if (pi Q (the desired resource by the controller) 
is optimal, then the proposed utility function generates a globally optimal outcome. 

9.3 Discussion 

Previously, we saw that the independent controller Control-B that incorporates global 
data, viz. the average temperatures, performs very well. In the present section we positively 
answered the question if one can construct a market, Market- B, that is based on local 
data only and that performs as well. 

For this result we have employed a market approach based on general equilibrium theory. 
This is of course not the only available mechanism for resource allocation in multi-agent 
systems. It seems interesting to try out other mechanisms, like the contract net protocol 
(Davis & Smith, 1983), and see if they perform better. However, Theorem 9.3 tells us 
that, if we treat the problem of building control as being separable in terms of agents (as 
also done by Huberman and Clearwater), there is no better scheme. 17 For example, if 
we assigned all the resources to an auctioneer, that on its turn would iteratively assign the 
total resource in small portions to bidders bidding with their true marginal utility, we would 
end up with something close to the competitive equilibrium, but we cannot do better than 
Market-B. Furthermore, this would be a computationally extremely inefficient way to 
arrive at equilibrium compared to other available methods (Ygge, 1998, Chapter 4). That 
is, we can use different mechanisms for achieving the competitive equilibrium, but we can 
never hope to find a mechanism that would do better than the Market-B scheme. 



17. We note that, as mentioned earlier, the problem is actually not fully separable in terms of agents and 
therefore better solutions may exist if this is taken into account. Another observation is that in this article 
we have investigated only the case of using the currently available resource as the interesting commodity, 
in accordance with the work of Huberman and Clearwater, and we found an optimal mechanism for that. 
We note however that extending the negotiations to future resources as well could potentially increase 
performance. But this is a different problem setting with different demands on available local and/or 
global information items, such as predictions. We have given a solution to this problem in recent work, 
see Ygge et al. (1999). 
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10. Conclusions 

We believe that both the approach and the results, as presented in this article, pose an 
interesting challenge to the software agent community. Multi-agent systems offer a new 
way of looking at information systems development, with a potentially large future im- 
pact. However, new approaches must prove their value in comparison and competition with 
existing, more established ones. The agent paradigm is no exception. 

We have therefore deliberately played the role of the devil's advocate in this article. In 
our view, a key question not yet satisfactorily answered by the software agent community 
is: in what respect and to what extent are multi-agent solutions better than their more 
conventional alternatives? This article has shown that arguing in favor of the multi-agent 
systems approach does require careful analysis beyond the disciplinary boundaries of com- 
puter science. Empirical and comparative studies in the multi-agent literature of the kind 
carried out in the present article are all too rare as yet. But as we have shown in techni- 
cal detail, established paradigms such as conventional central control cannot be that easily 
dismissed. A similar argument holds, by the way, regarding mathematical optimization 
techniques in distributed resource allocation problems, cf. our previous discussions (Ygge 
& Akkermans, 1996; Ygge, 1998). Many of them are, in a distributed guise, better than 
many newly proposed market protocols. 

Abstract considerations alone, concerning the general nature of agenthood, autonomy, 
rationality, or cooperation, are not sufficient to prove the value of the agent paradigm. Such 
theoretical reflections are worthwhile, but do not diminish the need for thorough analysis of 
(agent and market) failures and successes in real-life applications. Therefore, we have taken 
a different approach, aimed at obtaining experimental data points on the basis of which 
convincing software agent claims can be established. 

The data point considered in this article is climate control within a large building con- 
sisting of many offices. Given a measure, Eq. (1), a local controller approach, Eq. (11), 
current temperatures, and the setpoint temperature, we have investigated the problem on 
how to properly distribute the resource among the offices. This is a rather prototypical 
application relating to the general problem of optimal resource allocation in a highly dis- 
tributed environment. This class of problems has already received much attention in the 
multi-agent area. Reportedly, this type of application is very suitable for market-oriented 
programming (Huberman & Clearwater, 1995). On the other hand, we have devised some 
better conventional control engineering solutions, as well as alternative and better market 
designs. 

The main conclusions of our investigation are: 

• The market approach by Huberman & Clearwater (1994, 1995) indeed outperforms 
a standard control solution based on local, independent controllers. So, the market- 
based multi-agent approach indeed yields a working solution to this type of problem. 
Our analysis has shown that the success of this market approach depends on the 
agents communicating their local information to all other agents before the auction 
starts. 

• However, if conventional central control schemes are allowed to exploit the same global 
information, they perform even better. 
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• We have proposed an alternative market design based on general equilibrium theory 
that uses local data only. It performs better than the Huberman-Clearwater market 
and as well as a centralized control scheme having access to global information. 

• Our general conclusion can be formulated as a quasi-equation: "local information 
+ market communication = global control". This holds under suitable conditions 
(existence of market equilibrium, price-taking agents) and its validity has been specif- 
ically shown here for the case of building climate control (compare particularly the 
Control-B and the Market-B schemes we developed). However, as we intend to 
show in forthcoming work, this is a much more generally valid result. 

• The important difference is that in computational markets this global information is 
an emergent property rather than a presupposed concept, as it is in central control. 

In our analysis we have focused on the market approach. It is tempting to ask whether 
things are different when a non-market multi-agent approach is followed, say, using the 
contract net (Davis & Smith, 1983). As we argued, the answer in our opinion is a straight- 
forward no. The goal in the considered class of problems is to find the optimal distributed 
solution. Alternative agent approaches, market as well as non-market ones, only change the 
multi-agent dynamics on the way to this goal. This might be done in a better or poorer way, 
but it is not possible to change this goal itself. The goal state in any multi-agent approach 
is, however formulated, equivalent to market equilibrium, the yardstick for having achieved 
it is given by some quantitative performance measure as we discussed, and both are stable 
across different agent approaches. 

Thus, one of the main conclusions of this article is that one must be very careful when 
promoting multi-agent approaches to resource allocation problems for which traditional 
approaches are available. In particular, one should be cautious using arguments related 
to distribution of information (as was done by Huberman and Clearwater), or other com- 
putational aspects. Still, we argue that there are conceptual advantages of market-based 
approaches to this type of problems, as well as other advantages such as evaluation of local 
performance (cf. Section 2). These advantages do not show off very clearly in the present 
application setting - for example, it was assumed that all offices have the same thermal 
characteristics and office agents on the market are not added and deleted continuously on 
the fly - but they are clearly visible in other more general settings. 

A final note is that in this article we have devised an optimal market design given the 
problem formulation rather than devised an optimal approach to the general problem of 
building control. This is in line with the objective of this paper to give a comparative study 
of different possible approaches. That is, we have focused on a published and well-known 
problem formulation in order to focus on the subject of markets for resource allocation 
alone. As stated earlier in the article, several aspects can be improved, for example: 

• The local controllers. I-controllers are seldom used, rather PI- or PID-controllers are 
preferred. Furthermore, for these kind of computerized settings, modern digital con- 
trollers, based on e.g. pole-placement methods are preferable (Astrom &; Wittenmark, 
1990). 
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• The measure. The average value is at least as important as the standard deviation. 
(We have as a consequence excluded pathological solutions from our analysis by taking 
the average temperature as given in Theorem 9.3). 

• The incentives for office agents to reveal their true preferences in order to avoid spec- 
ulation. Reasonably, personnel in the offices should have the opportunity to make 
trade-offs between comfort and economical value. (It has been shown that generally 
speaking it is very difficult to benefit from speculation if the number of agents on the 
market is sufficiently large, and/or if uncertainty about the other agents' behavior 
exists (Sandholm k Ygge, 1997).) 

• Taking several time periods into account. As each office serves as a storage for heat, 
agents can for example gain by using relatively much resource during certain hours 
when the total resource need is small. 

That is, a more realistic approach to the building control problem is to use minimized 
cost as the measure, give the users real incentives to reveal their energy preferences (by 
letting them pay their actual energy costs so that attempted speculation will generally 
cost money), and take future time periods into account. Elsewhere we have solved the 
problem of dealing with simultaneous optimization over different time periods, by means of 
a multi-commodity market (Ygge et al., 1999). 

In such a more realistic and large-scale setting, the market-based approach is attractive 
compared to its alternatives. The abstractions used (prices and demands) are natural and 
easily understood by everyone, and they are uniform over all types of agents (even ones 
that do not use the resource for controlling a temperature). 

A major qualitative conclusion of this article is that "local data plus market communi- 
cation yields global control". This conclusion is based on the discussed application setting 
of building control, as an instance of a distributed resource allocation problem. It suggests 
that generally for this type of problem both central control engineering and multi-agent 
market solutions can be devised that give comparable optimal control quality. A next step 
is to prove this more generally in a rigorous mathematical fashion, delineating the precondi- 
tions in more detail. Such a proof can be based upon the (continuous and matrix-algebraic) 
dynamic systems theory available from control engineering, and especially upon results con- 
cerning what is called optimal control. We believe that this indeed can be done in a formal 
way, and as a conjecture we state the following general result. Multi-agent equilibrium mar- 
kets yield optimal decentralized solutions to distributed resource allocation problems that 
are of the same quality, in terms of a given overall systems performance index, as solutions 
given by the optimal central (multi-input, multi-output) systems controller that has access 
to all relevant local data. These local data involve the total system state and control vectors 
(in building control these are the vectors formed from the difference between the actual and 
setpoint temperatures for each office, and the cooling power for each office, respectively). 
Preconditions for this to be true are: (i) agents act competitively; (ii) the equilibrium ex- 
ists; (iii) the systems performance index can be written as a linear combination of local 
contributions (implying diagonality of certain matrices; if not, agents are not independent). 
These local contributions are directly related to the agents' utility functions. The agent 
approach will be more readily generalizable to large-scale systems and non-linear control 
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solutions (linearity is the main case where control engineering can get analytical mathe- 
matical expressions). This is indeed a strong and general statement about the relationship 
(and even outcome equivalence) between decentralized markets and central control. 
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Appendix A. Original Formulation of the Huberman-Clearwater Market 

All formulae in this section were directly taken from the papers by Huberman & Clearwa- 
ter (1994, 1995). 

Trade volumes First, the decision for an agent to buy or sell is based on 

_T^p_ (Tj) j t i0 >l, seller 
tl ° ~ T lo ' (T«*p) \ t lo < 1, buyer " 

Then, the total trade volume, V, is calculated from 

N 

Vi = Y,\l-Uo\, (22) 

o=l 

where N is the number of offices. 

Every agent calculates its request volume, v, according to 

Vio = Q ' ~ J %o][ ■ (23) 

When an agent buys or sells its v the actual movement of a valve, called VAV, is 
computed from 

AVAV io = f(flow io , v^, VAV io ), (24) 
after which the actual VAV position for each interval is updated according to 

VAV i+1>0 = VAV lo + AVAV io . (25) 

The function / in Eq. (24) stems from the physics of the office control, but it is not 
easy to derive in the general case and it is neither explicitly given in the original papers 
(Clearwater & Huberman, 1994; Huberman & Clearwater, 1995). A reasonable and simple 
choice is to assume a linear relationship (the cited papers suggest that they do so in practice 
as well), replacing Eq. (24) and Eq. (25) by 



= F i0 ± v io , (26) 



where plus or minus depend on whether it refers to an accepted buy or sell bid. P c i is 
obtained from Eq. (12). We have employed Eq. (26) in our simulations. The nature of this 
assumption is not crucial to our central line of reasoning. 

Bids The bids are based on a marginal utility 18 of the form described by 19 

U(t io /T^,m io ) = [f/(0, m , o )]( 1 - i -»^ t ") = [U(0,m to )f-^^\ (27) 



18. This is called utility in the work of Huberman and Clearwater. We prefer to use the term marginal utility 
instead, because it is directly related to price. This terminology conforms better to that of microeconomic 
theory. 

19. This notion of a utility function used by Clearwater and Huberman is very much based on the work of 
Steiglitz & Honig (1992) 
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with 

17(0, m lo ) = u 3 - {u 3 - Ul )e-^°, (28) 

and 



7 = In 



U 3 - U\ 



(29) 



_u 3 - u 2 _ 

where u\ = 20, u 2 = 200, and u 3 = 2000, and m is the amount of money that an agent has, 
given by 

m lo = 100(2 - VAV io ). (30) 
With the above relation between Fi and VAV, we rewrite Eq. (30) as 

rpmax 77 

m l0 = 100(2- » (31) 

1 o 

Observing Eqs. (28) and (29), we note that these equations can be simplified to 
U(0,m io ) = 113- (u 3 - ui)e~ im ™ = u 3 - (u 3 - ui)(e 7 )" 



«S-(«3-«l)(^)' 



(32) 



The bids are calculated from multiplying the marginal utility with the previous price, 
price, according to 

Bi t0 = Ui (t io /T* etp , m i0 ) ■ pricei. (33) 

This is the equation given by Huberman and Clearwater. Straightforward application 
of Eq. (33) turns out to produce major problems in simulations, however. Equation (27) 
shows that U(ti /T^ etp ,mi ) is minimized for minimized Tj G and maximized . As we 

can expect Tj G to be well above 10°C and to be well below 2, and U(0,m) s=s 2000, 

we can be sure that U(ti /T* etp , mj ) will be well above 2000 1_ ^J « 437. Thus, the bidding 
price will never be below 437. Then, Eq. (33) tells us that the market price will be at 
least priceo ■ 437*. This leads to numerical overflow after a few iterations. We note however 
that, since all agents multiply their bids with the previous price, this has no effect on the 
reallocation itself: it affects only the price level. Therefore, we omit multiplying by the 
previous price in our simulations, so that the agents' bid prices in Eq. (33) equal their 
marginal utilities of Eq. (27). Huberman and Clearwater state that they burn all money 
after each auction round in order to avoid inflation and overflow, indicating that they may 
adopt a similar procedure. In any case, this procedure leads to exactly the same allocations 
but avoids numerical overflow. 
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